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ABSTRACT 


The  Calgary  Public  School  Board,  in  May  1957, 
accepted  the  proposal  that  principals  of  elementary  and 
elementary- junior  high  schools  be  given  more  released  time 
for  supervision.  At  the  same  time  they  requested  that  a 
study  be  made  to  ascertain  the  value  of  this  plan  of 
de-centralized  supervision  over  a  two-year  period.  A 
committee  of  supervising  principals  and  central-office 
personnel  directly  concerned  with  instruction  was  named 
to  carry  out  the  investigation. 

The  standard  on  which  the  evaluation  was  made 
consisted  of  an  Ideal  List  of  sixteen  specific  factors 
that  contribute  to  the  effectiveness  of  instruction  in 
the  classroom  and  which  can  be  improved  by  supervisory 
assistance.  This  list  was  compiled  from  opinionnaires 
submitted  by  the  principals.  The  rank  order  of  the  factors 
was  held  secret  throughout  the  experiment. 

A  Rating  Scale,  based  on  the  Ideal  List,  was  designed 
to  assess  the  effectiveness  of  instruction  in  the  classroom. 

In  order  to  evaluate  the  scheme  principals f  ratings 
of  teachers1  classroom  instruction  were  made  in  1957  and 
again  in  1959.  A  comparison  of  rank-difference  correlations 
between  the  Ideal  List  and  the  Teacher  Rating  determined  in 
1957  with  the  correlations  taken  in  1959  would,  it  was  felt, 
be  an  indication  of  the  effectiveness  of  the  plan. 
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Investigation  showed  that  the  correlation  between 
the  1957  Ideal  List  and  the  1957  Teacher  Rating  to  be  .LI. 
The  correlation  between  the  1957  Ideal  List  and  the  1959 
Teacher  Rating  was  .53.  The  increase  in  the  size  of  the 
correlation,  it  was  concluded,  indicated  some  degree  of 
success  for  the  plan  was  apparent. 

During  the  two-year  period  between  1957  and  1959 
a  variety  of  factors  such  as:  in-service  training  courses, 
activities  of  the  Supervising  Principals’  Association, 

V 

discussion  groups,  individual  reading,  and  staff  meetings, 
had  an  effect  on  the  principals’  concept  of  supervision. 
Therefore,  a  revised  ranking  of  the  1957  Ideal  List  was 
made . 

Investigation  then  showed  that  the  correlation 
between  the  1959  Ideal  List  and  the  1959  Teacher  Rating 
to  be  .37.  This  large  correlation,  when  compared  with 
that  of  1957,  was  felt  to  be  indicative  of  considerable 
growth  in  the  effectiveness  of  supervision. 

It  should  be  noted  that  a  similar  but  smaller 
growth  was  indicated  when  the  1957  Ideal  List  was  used  as 
the  standard  on  which  change  was  measured  but  that  no 
growth  was  indicated  when  the  1959  Ideal  List  was  used 
as  the  standard. 

It  is  suggested  that  further  study  be  made  of  all 
aspects  of  the  supervisory  program  before  complete 
evaluation  is  attempted. 
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CHAPTER  I 


THE  PROBLEM 

I.  INTRODUCTION 

One  of  the  major  problems  currently  facing  school 
boards  at  home  and  abroad  is  an  exploding  school 
population.  New  schools,  new  ideas,  and  expanding 
programs  along  with  growing  demands  for  improved  standards 
harass  continuously  those  responsible  for  the  provision 
of  educational  facilities.  A  mere  expansion  of  existing 
facilities  and  organization  has  not  been  the  answer  to 
growing  demands  being  placed  upon  the  educational  insti¬ 
tution.  Rather  wide-spread  centralization  of  governing 
bodies  and  school  facilities,  and  sweeping  reorganization 
of  administrative  programs  are  being  instituted  experi¬ 
mentally. 

Recognizing  the  dilemma  facing  them,  school 
boards  and  their  administrative  officers  have  sought 
solutions  to  their  problems  from  their  own  experience, 
that  of  others,  and/or  the  application  of  new  and 
advanced  theories  emanating  from  educational  experts. 
Because  of  the  uniqueness  of  the  situation  in  which  each 
board  finds  itself  and  because  of  the  speed  with  which 
the  explosion  has  struck,  much  trial  and  error  and 
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on-the-spot  action  research  is  taking  place.  Such  was 
the  situation  faced  by  the  Calgary  Public  School  Board 
in  the  Spring  of  1957  with  regard  to  supervision  of  its 
teaching  staff. 


II.  STATEMENT  OF  THE  PROBLEM 

The  purpose  of  this  study  was  to  evaluate  the 
de-centralized  plan  of  supervision  as  practiced  in  the 
Calgary  Public  Elementary  and  Junior  High  Schools. 

The  evaluation  was  carried  out  with  the  following 
question  in  mind: 

Is  time  free  from  teaching  duties  for  supervising 
principals  effective  in  improving  the  teaching-learning 
situation  in  the  classroom? 

III.  BACKGROUND  OF  THE  STUDY 

Less  than  a  decade  following  the  cessation  of 
hostilities  of  World  War  II  the  effects  of  the  expanding 
school  population  was  upon  the  Calgary  school  system. 

In  the  period  from  the  end  of  the  war  to  1957  the  school 
population  and  the  teaching  staff  had  more  than  doubled 
and  it  was  apparent  that  major  reorganizational  steps 
must  be  taken  in  nearly  every  phase  of  the  system.  This 
study  will  confine  itself  to  one  particular  area  of 
reorganization,  namely,  that  of  supervision. 
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Although  many  fine  lines  of  demarkation  are  drawn 
between  administration  and  supervision  in  educational 
literature,  the  term  supervision  as  used  in  this  study 
shall  refer  to  those  activities  which  are  planned  to 
assist  in  the  development  of  an  improved  teaching¬ 
learning  situation  in  the  classroom. 

The  question  of  providing  adequate  supervision 
for  the  maintenance  and  improvement  of  classroom 
instruction  became  a  matter  of  growing  concern  in  the 
Calgary  system.  Two  alternatives  were  considered  by  the 
superintendent  and  his  assistants.  The  first  was  to 
expand  the  central-office  supervisory  staff  by  the 
appointment  of  more  supervisors  and  assistants.  This 
body  of  supervisors  would  carry  on  its  work  from  the 
central  office.  The  second  alternative,  and  one  which 
had  been  given  a  limited  trial,  was  to  make  time 
available  to  principals  so  that  they  could  perform  the 
necessary  supervisory  functions  in  their  schools.  In 
either  case,  the  purpose  would  be  to  provide  teachers 
with  the  assistance  needed  to  carry  out  and  improve  the 
calibre  of  instruction  being  provided  in  the  classroom. 

The  central  administrative  staff  favored  the 
second  plan.  This  scheme  of  de-centralized  supervision 
was  presented  in  the  Spring  of  1957  to  the  Calgary  Publi 
School  Board  and  accepted  by  them  on  a  tentative  basis. 


v. 


L  .  :  '  '  r  ■  ''r  ’  ■  '  i  . 

o  •  .  i:  . ;t  ,  O  '  •  ■  ' 

■  M  '  • 

rr?'  'Oo  l  vo'i  '  d  i  £.  ■  >  Jr  ■  •  ol  vc<;  •  eiM  i  i  dui.?,e;'. 

t  :  >■.  '  •: 


,  [oi  v  oiJ&'&up  ed T 

)0'  1  .  .  ■■  ■  ll  bn  ’  ■  5£Jl  ’  :  o':-'.' 

'i  o:  oo  J.  to  ■  ■  no  •  ;•  ■  '■  o  '  o  o ‘  ::  o  :  o' ;  J: 

. 

:  ■  Jo.'-  u 

■  .j-.  00  ’  ~.r  Zll-  00  '0  ' .t  "  O'TS 

.  ''  "  .  J'  .  "  '  ’■ 

moil  2f*iow  r  no  \;ri£o  bisiow  aic  sivt  '  c 

. 


t-  ■ 

9fft  1 :  ■  '  "  ■  ■  0  d  I  o  :  '  ■  £  Id®,  j  BV 

' 

< 

6  r  •  1  I  3  c  o  '  •'  0  ' 


•"  j  ‘  .  '  . :  '  o'  >0 

"‘.I  •:  •’  J"  •  •  I  o  0 

to  :  *iq  i 

•  '  '  '  -  ■  J 


4 


The  Board  agreed  to  authorize  an  increase  in  time  alloted 
to  principals  for  supervisory  activities  and  at  the  same 
time  directed  that  an  evaluation  of  supervision  be 
undertaken  over  the  next  two  years. 

Following  a  period  of  planning  and  preparation  the 
actual  evaluation  was  carried  out  between  September  1957 
and  February  I960.  During  this  time  an  attempt,  as  here 
reported,  was  made  to  evaluate  the  de- centralized  system 
of  supervision  as  approved  by  the  Board. 

No  attempt  will  be  made  to  determine  whether  the 
amount  of  time  set  by  the  Board  was  adequate.  The 
increased  time  for  supervision  was  granted  on  a  scale 
based  on  the  number  of  rooms  supervised.  By  measuring 
the  situation  as  it  existed  in  1957  and  again  in  1959 
an  attempt  will  be  made  to  determine  the  effect  of  time 
free  for  supervision  on  classroom  instruction. 

IV.  DEFINITION  OF  TERMS 

Supervision.  Those  activities  which  are  planned 
to  assist  teachers  in  the  development  and  improvement  of 
the  teaching-learning  situation  in  the  classroom. 

De-centralized  Supervision.  Supervision  carried 
on  in  the  school  by  the  supervising  principal  as  opposed 
to  supervision  by  supervisors  from  central  office. 
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Central  Office  Administrative  Staff.  The 
superintendent,  assistant  superintendents,  and  such 
supervisors  and  assistants  not  in  charge  of  specific 
schools  but  carrying  out  duties  from  the  central  office 
in  all  schools  or  particular  divisions  of  the  system. 

Time  Free  For  Supervision.  Time  free  from 
teaching  duties  to  carry  out  supervisory  activities. 

Ideal  List .  The  consensus  of  supervising  principals 
as  to  what  constitutes  sixteen  important  specific  charac¬ 
teristics  of  effective  classroom  instruction  which  can 
be  improved  by  supervisory  assistance. 

V.  LIMITATIONS  OF  THE  STUDY 

The  present  study  was  confined  to  an  evaluation 
of  supervision  as  practiced  in  the  City  of  Calgary 
Public  Elementary  and  Junior  High  Schools  during  the 
period  September  1957  to  February  i960.  The  personnel 
involved  in  the  survey  included  all  principals  allotted 
time  free  from  classroom  duties  for  the  purpose  of 
supervision  and  all  teachers  teaching  more  than  half  time 
in  these  particular  schools. 

The  success  of  the  de-centralization  of  supervision 
could  be  measured  in  various  ways.  Its  effect  on  pupil 
achievement  in  particular  subject  fields  might  have  been 
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measured  by  using  matched  groups  under  centralized  and 
under  de-centralized  programs.  It  is  possible  that  a 
carefully  planned  supervisory  program  assigned  to  a 
definite  group  of  principals  would  have  shown  greater  or 
lesser  advantages  or  disadvantages  over  a  non-directed 
group.  Under  the  existing  circumstances  and  conditions 
evaluation  of  whether  or  not  time  free  for  supervision 
was  effective  appeared  most  urgent  and,  therefore,  that 
phase  of  supervision  received  attention. 

The  gross  picture  thus  obtained  was  perhaps  the 
initial  step  toward  more  refined  proceedings  which  are 
without  doubt  worthy  of  consideration  but  were  not 
examined  in  this  study. 

All  conclusions  drawn  concerning  the  effectivene 
of  de-centralized  supervision  were,  therefore,  based  on 
whether  improvement  in  the  teacher-learning  situation 
was  or  was  not  obtained  from  additional  time  granted 
principals  for  supervisory  activities.  An  evaluation 
including  a  wider  investigation  of  the  many  existing 
variables  could  be  considered  for  further  study. 

VI.  HYPOTHESES 

1.  The  basic  hypothesis  for  this  study  was  that 
time  given  to  principals  for  supervision  would  cause  a 
positive  and  significant  growth  in  the  effectiveness  of 
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classroom  instruction. 

2.  Individual  principals’  ratings  of  classroom 
instruction  will  parallel  more  closely  their  consensus  of 
an  ideal  situation  at  the  close  of  the  study  than  they  do 
at  the  beginning. 


VII.  BASIC  ASSUMPTIONS 

1.  The  background  and  experience  of  principals 
guarantees  the  validity  of  an  ideal  list  of  factors  to 
be  employed  as  the  criterion  for  measuring  change  in  the 
teaching-learning  situation. 

2.  Supervising  principals  are  capable  of  rating 
classroom  instruction  and  will  submit  an  honest  picture 
of  existing  practices  when  asked  to  do  so. 

3.  Increased  time  for  supervision  will  be  utilized 
by  principals  for  assisting  teachers  to  improve  instruction. 

4.  The  rank  order  of  the  ideal  list  of  charac¬ 
teristics  of  classroom  instruction  which  can  be  improved 
by  supervisory  assistance  will  not  change  significantly 
over  the  two-year  period. 
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RELATED  LITERATURE 

Although  many  volumes  have  been  written  in  the  field 
of  education  on  evaluation  and  on  supervision,  investigation 
soon  reveals  that  scientifically  developed  instruments  for 
evaluating  supervision  are  almost  unknown.  More  interest 
seems  to  have  been  paid  to  the  development  of  programs  of 
activities  for  improved  supervision  than  to  their  evalu¬ 
ation.  Desirable  as  these  programs  may  be,  something  is 
lacking  if  all  there  is  to  support  them  is  a  general 
impression  or  an  individuals  personal  experience. 
Nevertheless,  it  must  be  recognized  that  unanimity  of 
opinion  does  increase  confidence  in  them.  That  such  is 
the  case  is  supported  by  the  following  statement  made  by 
Barr,  Burton  and  Brueckner  in  a  193$  publication^  and 
repeated  verbatum  by  Burton  and  Brueckner  nearly  twenty 
years  later  in  a  1955  publication: 

...  we  find  ourselves  in  the  position  of 
having  reported  in  the  literature  of  education 
many  kinds  of  improvement  programs  and  activities 
recommended  by  various  members  of  the  school 


A.  S.  Barr,  William  H.  Burton,  and  Leo  J. 
Brueckner,  Supervision.  New  York:  D.  Appleton-Century 
Company,  1938,  p.  798. 
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personnel,  on  the  basis  of  their  own  experience, 
but  without  scientific  validation.2 

I.  PURPOSES  FOR  WHICH  EFFECTIVENESS  OF  SUPERVISION 

MAY  BE  EVALUATED 

There  continually  exists  in  the  life  of  any  school 
system  the  demand  for  justification  of  activities  and 
expenditures.  Research  would  indicate  that  more  often 
than  not  support  is  gained  by  the  expounding  of  untested 
theories  or  the  quoting  of  judgments  based  on  personal 
experience.  Administrators  have,  by  such  methods, 
guided  their  systems,  rightly  or  wrongly,  along  apparently 
successful  paths.  But,  the  ever  increasing  demands,  both 
from  within  and  without  the  field  of  education,  for 
scientific  validation  of  programs  and  activities  are 
placing  upon  administrators  of  school  systems  the  ever 
increasing  responsibility  of  justifying  both  existing 
and  suggested  changes  on  scientific  research.  Though 
evaluation  is  not  something  new  the  areas  of  its  use  are 
expanding  and  its  methods  becoming  more  refined. 

As  viewed  by  Barr,  Burton  and  Brueckner  evaluation 
of  the  effectiveness  of  supervision  has  two  purposes. 3 

o 

William  H.  Burton  and  Leo  J.  Brueckner,  Supervision 
A  Social  Process .  New  York:  Appleton-Century-Crofts,  Inc., 
1955,  p.  WT. 

3 

^Barr,  Burton  and  Brueckner,  ojd.  cit . ,  p.  799. 
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The  first  purpose  is  that  of  justifying  the  cost  of  this 
aspect  of  the  program  in  general  and  the  second  purpose 
is  to  supply  valuable  information  about  the  effectiveness 
of  particular  supervisory  devices  employed  in  improving 
the  conditions  for  more  effective  learning  and  teaching. 

Morphet  summarizes  the  need  for  evaluation  in  six 
statements: 

1.  Parents  and  citizens  want  to  know  how  good 
their  schools  are. 

2.  Board  policies  are  continued,  established, 
and  revised  on  such  judgments. 

3.  Administrative  recommendations  are  based  on 
appraisals. 

4.  Teachers  (Supervisors)*  are  interested  in 
the  results  of  their  efforts. 

5.  Teacher  (Administrative)*  morale  is  highly 
related  to  the  understanding  or  beliefs  they  have 
regarding  the  functioning  of  their  systems. 

6.  Judgments  and  criticisms  may  be  effectively 
countered  by  adequate  facts.**- 


II.  IMPORTANCE  OF  EVALUATION 


Hicks  stresses  the  importance  of  evaluating  all 
phases  of  an  educational  system  in  the  following 
quotation: 

No  human  enterprise  deserves  support  unless 
there  is  some  form  of  demonstrated  evidence  of  its 


*Inserted  by  writer. 

^Johns  Reller  Morphet,  Educational  Administration. 
Englewood  Cliffs,  N.J.:  Prentice-Hall,  Inc.,  1959, 
pp.  524-525. 
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value  in  the  achievement  of  worthwhile  purposes. 

Since  the  school  is  a  direct  instrument  of  society 
depending  largely  upon  public  support,  it  is 
particularly  imperative  that  educators  engage  in 
systematic  educational  inventories  which  will  reveal 
the  nature  and  degree  of  the  schools’  success. 5 

In  the  words  of  John  Morphet  the  results  of 
evaluating  programs  are  purposeful: 

1.  To  secure  the  basis  for  making  judgments 
at  the  end  of  a  period  of  operation. 

2.  To  insure  continuous,  effective,  and  improved 
operation. 

3.  To  diagnose  difficulties  and  avoid  destructive 
upheavals. 

4.  To  improve  staff  and  citizen  ability  to  develop 
the  educational  system. 

5.  To  test  new  approaches  to  problems  and  to 
conduct  pilot  studies  in  the  consideration  of  which 
advancements  can  be  effected. 

Any  evidence  of  quality  gained  through  evaluation 
processes  is  advertising  a  school  system  can  enjoy  and 
is  an  excellent  basis  for  defense  against  criticism. 
Continuous  emphasis  on  evaluation  tends  to  keep  a  system 
dynamic  and  the  people  in  it  professionally  alive. 
Effective  evaluation  is  an  essential  prerequisite  to 
constructive  change  and  improvement. 


^Hanna  J.  Hicks,  Administrative  Leadership  in  the 
Elementary  School.  New  York:  The  Ronald  Press  Company, 
1956,  p.  405. 

Viorphet,  02.  cit . .  pp.  525-527. 
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That  the  appraisal  of  supervision  is  desirable  is 
supported  by  Barr  and  Brueckner  in  their  estimate  of  the 
outcomes  of  evaluation: 

Just  as  teachers  and  pupils  have  profited  both 
directly  and  indirectly  from  the  introduction  of 
more  accurate  methods  of  measurement  into  the  realm 
of  teacher  and  pupil  growth,  there  is  every  reason  to 
believe  that  supervisors  and  administrators  too 
would  profit  by  the  introduction  of  similar  means  of 
evaluation  into  their  own  work  as  school  leaders. 
Every  person  with  leadership  responsibility  should 
be  expected  to  furnish  tangible  evidence  of  the 
effectiveness  of  the  improvement  programs  that  he 
proposes  and  puts  into  operation.' 


III.  THE  BASIS  OF  EVALUATION 

There  is  general  agreement  among  writers  in  the 
area  of  evaluation  that  there  must  be  a  starting  point, 
a  criterion,  a  standard,  either  existing  or  established, 
against  which  progress  can  be  measured.  These  standards 
may  exist  or  be  established  for  a  complete  area  of 
investigation  or  for  a  particular  phase  of  the  area. 

In  attempting  to  evaluate  the  effectiveness  of  supervision 
there  are  so  many  variables,  conditions  and  circumstances 
controlling  and  contributing  to  the  success  or  failure  of 
the  measurement  that  encompassing  investigations  are 
rare  indeed.  Nevertheless,  many  attempts  have  been 
launched  to  gain  valid  knowledge  in  limited  areas.  As 

^Burton  and  Brueckner,  o£.  cit . ,  p.  655. 
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these  specific  areas  are  identified  with  supervision  so 
is  the  effectiveness  of  supervision  measured.^ 

In  some  way,  then,  a  means  of  measuring  outcomes 
should  be  sought  if  one  is  to  determine  the  effectiveness 
of  supervision. 


IV.  METHODS  COMMONLY  EMPLOYED 
IN  THE  EVALUATION  OF  THE  EFFICIENCY 
OF  SUPERVISORS  AND  SUPERVISION 


The  complexity  of  the  situation  which  arises  when 
a  choice  of  method  for  evaluating  is  being  made  is 
clearly  illustrated  by  the  observation  made  by  a  group 
of  writers: 

Just  as  we  determine  the  effectiveness  of  a 
given  teacher,  so  we  should  evaluate  the  effective¬ 
ness  of  the  supervisory  program  by  measuring  its 
outcomes  in  some  way.  Although  its  actual  value 
is  difficult  to  determine  since  the  complex  situation 
in  which  it  is  embedded  contains  many  associated 
factors  whose  effects  are  difficult  to  control  or 
to  measure. 9 

An  analysis  of  the  literature  on  the  evaluation 
of  supervision  would  indicate  that  the  most  commonly 
employed  methods  of  evaluating  are  those  which  involve 
measurement  of  pupil  growth  and  achievement  and  those 


^Burton  and  Brueckner,  0£.  cit . ,  p.  656. 
9lbid. 
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involving  evaluation  of  a  particular  supervisor  or 
supervisory  program.  It  would  appear  that  the  former 
method  is  more  generally  used,  not  because  it  is 
necessarily  better  than  others,  but  because  means  of 
testing  pupil  growth  are  so  readily  available  in  the 
form  of  standardized  tests  and  established  norms. 

Studies  involving  the  measurement  of  pupil  growth 
and  achievement  have  been  employed  in  many  ways  but  in 
general  they  could  be  classified  as  either  uncontrolled  or 
controlled  measurements.  Writers  feel  that  uncontrolled 
measures  of  the  products  of  learning  constitute  fairly 
reliable  measures  of  the  effectiveness  of  supervision  if 
the  measurements  are  extended  over  a  considerable  period 
of  time  and  if  the  factors  conditioning  the  products 
measured  are  within  the  control  of  the  supervisor  or 
supervisors  to  be  evaluated.  The  degree  of  control  over 
the  many  factors  which  combine  to  effect  change  and  results 
determines  the  degree  of  validity  to  be  found  in  such 
evaluations . 

Against  a  criterion  of  what  is  expected,  measures 
are  made.  In  the  absence  of  complete  control  over  related 
factors  it  is  possible,  nevertheless,  to  secure  some 
ideas  of  the  efficiency  of  one’s  work  by  comparing  the 
achievement  of  the  pupils  of  one’s  own  subject,  school, 
or  school  system  with  that  of  the  pupils  in  a  similar 
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situation.  Though  not  without  very  definite  limitations, 
objective  data  of  this  sort  may,  when  the  comparisons  can 
be  made  between  comparable  groups,  constitute  a  valuable 
indication  of  the  effectiveness  of  particular  supervisors 
and  supervisory  programs. 

Controlled  studies,  in  contrast  to  the  uncontrolled, 
because  of  the  exacting  demands  made  upon  them  to  regulate 
all  aspects  affecting  learning,  tend  to  narrow  their  areas 
of  research  to  specific  aspects  of  particular  supervisory 
programs.  Investigators  in  this  area  generally  select  a 
particular  subject  or  aspect  of  a  program  and  then  apply 
a  defined  supervisory  practice  for  a  specific  period  of 
time.  They  then  assume  that  any  change  measured  is  a  direct 
result  of  the  practice  applied  and  is  an  estimate  of  the 
effectiveness  of  such  a  practice.  Although  such  a  procedure 
of  measuring  efficiency  is  by  no  means  perfect  or  all- 
inclusive,  writers  agree  that  because  of  its  objectivity 
and  attempted  control  it  is  superior  to  general  global  and 
merit  ratings  and  the  uncontrolled  types  of  measurement. 

It  would  appear  that  as  researchers  employ  and  refine  these 
limited  evaluative  instruments  and  methods  succeeding 
procedures  will  become  more  and  more  encompassing  and 
valuable . 

Probably  the  oldest  method  of  evaluating  supervisory 
effectiveness  is  the  use  of  the  check  list.  It  attempts 
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to  rate  qualities  commonly  associated  with  efficiency. 

The  chief  advantage  reported  for  the  use  of  such  lists  is 
the  stimulation  they  offer  those  rated  for  professional 
growth.  The  most  frequently  mentioned  disadvantage  is 
that  the  use  of  such  procedures  creates  undesirable  attitudes 
in  the  minds  of  those  rated. 
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CHAPTER  III 


THE  PROCEDURE 

This  chapter  explains  the  preparation  of  the 
Ideal  List  and  its  subsequent  revision,  the  construction 
of  the  Teacher  Rating  Scale  and  its  application,  and  the 
method  of  correlating  the  results  of  the  survey. 

I .  THE  TASK 

The  task  in  this  study  was  to  evaluate  a  particu¬ 
lar  plan  of  de-centralized  supervision.  In  the  plan 
principals  were  given  time  free  from  classroom  duties  to 
carry  out  supervisory  duties  which  had  formerly  been  almost 
exclusively  conducted  by  the  central  office  supervisory 
staff.  Relieving  teachers  were  employed  to  fill  in  for 
principals. 

After  due  deliberations  by  the  evaluation  committee 
named  to  carry  out  the  project  two  major  decisions  were 
made.  An  instrument  must  be  devised  or  found  to  measure 
any  change  that  took  place  in  the  effectiveness  of  class¬ 
room  instruction.  Recognizing  that  the  control  of  all 
variables  in  such  an  evaluation  was  beyond  the  realm  of 
possibility,  under  the  existing  circumstances,  it  was 
assumed  that  any  change  in  the  effectiveness  of  super¬ 
vision  would  be  due  to  the  major  variable — time  free  for 
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supervision. 

The  combined  knowledge  and  experience  of  the 
committee  did  not  reveal  the  existence  of  an  instrument 
that  could  be  used  for  the  experiment  so  it  remained 
with  them  to  construct  one. 

A  modified  rating  scale  of  employee  characteristics 
devised  by  Dr.  C.  Safran  for  use  in  business  for  rating 
employees  was  proffered  as  a  model  for  the  teacher  rating 
scale.*  Statements  in  it  describing  work  habits  were 
replaced  by  statements  describing  good  teaching  habits 
and  procedures.  The  scale  was  modified  to  rate  teachers. 
Supervising  principals,  following  classroom  visits  and 
general  observation  of  the  teacher  at  work,  were  to  arrange 
the  statements  in  descending  order,  ranging  from  the  most 
to  the  least,  as  they  appeared  in  their  judgment  to  be 
applicable. 


II.  PREPARATION  OF  THE  IDEAL  LIST 

Since  all  supervising  principals  made  regular 
classroom  visits  to  determine  where  assistance  was  needed 
it  was  decided  that  a  list  of  characteristics  of  good 
classroom  instruction  which  could  be  improved  by  super¬ 
visory  assistance  could  be  adapted  to  the  rating  scale  to 
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measure  any  change  in  the  teaching-learning  situation. 

This  list  was  to  be  known  as  the  Ideal  List. 

To  accommodate  the  rating  scale  which  employed 
sixteen  of  the  most  important  characteristics  of  class¬ 
room  instruction  an  Ideal  List  was  constructed. 

To  determine  the  content  of  the  Ideal  List  the 
knowledge  and  background  of  local  principals  was  employed. 
Using  this  source  and  not  outside  authority  made  the  list 
unique  to  Calgary.  The  requirements  for  appointment  to 
the  position  of  principal  in  the  Calgary  system  included  a 
minimum  of  a  Bachelor’s  Degree,  at  least  two  years  ex¬ 
perience  in  the  Division  in  which  the  appointment  was  made, 
a  period  of  in-service  training  as  a  vice-principal,  and 
several  years  of  successful  teaching  experience.  These 
requirements,  it  was  felt,  supported  the  confidence  placed 
in  local  principals’  opinions  as  to  what  constituted  good 
classroom  instruction.  The  consensus  of  such  a  group  would 
be  further  validated  by  its  numbers — approximately  sixty. 
Finally,  since  the  analysis  of  the  submissions  was  to  be 
scrutinized  and  judged  by  a  panel  of  nine  experienced 
experts,  including  superintendents,  supervisors  of  in¬ 
struction,  and  supervising  principals  actively  engaged  in 
the  field  of  education,  the  validity  of  the  Ideal  List 
would  be  further  supported. 

The  principals’  opinions  were  submitted  on  a  form 
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requesting  them  to  list  in  descending  order  of  importance 
sixteen  specific  factors  that  contribute  to  the  effective¬ 
ness  of  instruction  in  the  classroom  and  which  can  be 
improved  by  supervisory  assistance.* 

Upon  receipt  of  the  opinionnaires  from  the  principals 
a  tabulation  of  the  specific  factors  by  means  of  a  frequency 
distribution  was  made.  Because  an  ordering  of  the  points 
submitted  was  given,  a  system  of  weighting  was  devised  so 
that  first  choices  carried  more  importance  than  second 
choices,  second  choices  more  than  third  choices,  and  so  on. 
The  combined  frequency  and  weight  assigned  each  factor 
determined  its  relative  rank  and  the  first  sixteen  consti¬ 
tuted  the  Ideal  List. 

Aware  that  a  common  vocabulary  might  not  be  forth¬ 
coming  in  the  submissions  it  was  accepted  that  the  final 
interpretation  of  factors  would  be  placed  in  the  hands 
of  the  evaluation  committee.  The  result  of  such  a  procedure, 
it  was  anticipated,  would  produce  two  things:  a  list  of 
key  words  distinguishing  each  factor,  and  a  list  of  accom¬ 
panying  statements--one  for  each  characteristic — that 
would  give  a  clear  explanation  of  the  area  covered  by  each 
factor.  Every  attempt  was  made  to  make  the  items  as 
mutually  exclusive  as  possible. 
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Having  arrived  at  this  plan  of  action  and  in 
preparation  for  the  scrutiny  of  the  submitted  lists  the 
evaluation  committee  explored  and  discussed  a  possible 
ideal  list.  It  was  felt  that  such  a  discussion  would 
better  prepare  the  committee  for  the  interpretation  of 
the  submissions. 

Thus,  the  first  step--an  ideal  list  of  sixteen 
specific  factors  that  contribute  to  the  effectiveness  of 
instruction  in  the  classroom  and  which  can  be  improved  by 
supervisory  assistance — was  carried  out. 

III.  CONSTRUCTION  OF  THE  RATING  SCALE 

Testing  the  hypothesis  required  an  instrument 
that  would  measure  the  change  in  classroom  instruction 
that  might  take  place.  The  plan  called  for  a  rating  of 
instruction  as  it  existed  in  1957  and  again  in  1959. 

This  was  accomplished  by  the  use  of  the  rating  scale 
described  below.* 

As  stated  on  page  20,  the  final  form  of  the  Ideal 
List  consisted  of  sixteen  key  words  each  followed  by  a 
statement  of  explanation.  The  factors  were  listed  as 
statements  on  a  form  but  not  in  their  order  of  importance 


*Appendix — pp.  55-56. 
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This  list  was  followed  by  a  rating  form  as  illus¬ 
trated  below: 

Statement  is  most  applicable 
Statement  is  very  applicable 
Statement  is  applicable 
Statement  is  moderately  applicable 
Statement  is  less  applicable 
Statement  is  still  less  applicable 
Statement  is  least  applicable 

Supervising  principals  were  asked  to  visit  the 
classrooms  in  their  respective  schools  and  to  complete 
one  rating  form  for  each  teacher  observed.  When  the  Ideal 
List  opinionnaires  were  distributed  their  later  use  in 
the  rating  scale  was  explained  so  that  principals  would 
give  some  thought  to  the  ratings  they  would  be  asked  to 
make  at  a  subsequent  date. 

To  complete  the  rating  scale  the  principal  placed 
the  number  of  the  statement  he  found  to  be  most  applicable 
to  the  particular  teacher  under  observation  in  the  first 
square.  Recognizing  that  the  ordering  of  the  sixteen 
statements  in  a  continuum  would  be  most  difficult,  only 
the  most  applicable  and  the  least  applicable  were  to  stand 
alone.  Therefore,  to  facilitate  the  assigning  of  the  less 
distinguishing  statements  no  difference  in  degree  was 
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expected  between  the  second  and  third  choices  nor  between 
the  fourteenth  and  fifteenth  choices,  and  in  similar  manner 
in  the  remaining  choices. 

Two  precautions  ensured  an  accurate  and  unbiased 
picture  of  the  instruction  as  it  existed  at  the  time  of 
the  ratings.  First,  the  rank  order  of  the  Ideal  List  was 
held  in  camera  by  the  committee  throughout  the  duration  of 
the  evaluation  so  that  principals  would  not  be  biased  in 
their  ratings  by  it.  And,  second,  all  teacher  ratings 
were  annonymous  so  that  any  fears  on  the  part  of  the 
teacher  of  being  individually  rated  were  allayed  and  full 
cooperation  in  the  experiment  resulted. 

IV.  THE  METHOD 

In  the  compilation  of  the  Ideal  List  the  differences 
among  the  factors  were  expressed  by  ranking  them  in  a  one- 
two-three  order  of  importance.  With  such  an  arrangement 
in  order  of  merit  the  computation  of  the  correlation 
between  any  two  of  the  series  established  relied  upon  the 
rank-difference  correlation  method  of  comparison.  Although, 
as  stated  by  Garret,^  coefficients  of  correlation  calculated 
from  a  few  cases  are  not  very  reliable  at  best,  the  rank- 

^Henry  E.  Garrett,  Statistics  In  Psychology  and 
Education.  Toronto:  Longmans,  Green  and  Co.,  1957,  p.  354. 
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difference  method  will  give  as  adequate  a  result  as  that 
obtained  by  a  more  refined  technique.  The  large  number 
of  cases  used  to  arrive  at  the  rank  order  in  this  study 
tends  to  increase  the  validity  of  the  resulting 
correlations.  Therefore,  in  testing  the  hypothesis  in  the 
study  the  method  of  rank-difference  was  employed. 

V.  THE  INTERVENING  PERIOD 

During  the  two-year  period  between  the  initial 
run  and  the  final  run,  steps  were  taken  to  ensure  more 
uniform  practices  among  supervising  principals, 
especially  among  new  appointees.  Newly  appointed  super¬ 
vising  principals  were  given  a  series  of  informative 
lectures  on  the  elementary  curriculum  and  its  application 
in  the  Calgary  School  system.  These  lectures  were  prepared 
and  conducted  by  the  central-office  supervisory  staff. 
Experienced  supervising  principals — those  with  more  than 
three  years’  experience — divided  themselves  into  dis¬ 
cussion  groups  to  take  a  close  look  at  supervisory  problems 
and  procedures.  Both  of  the  above  practices  were  aimed 
at  a  more  uniform  and  economical  use  of  supervisory  time. 

It  was  believed  that  such  a  program  tended  to  improve 
the  validity  of  the  estimates  and  ratings  contributed  by 
all  principals. 
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VI.  REVISION  OF  THE  IDEAL  LIST 


In  order  to  ascertain  any  change  that  had  taken 
place  in  the  opinions  of  supervising  principals  as  to  the 
ranking  of  the  sixteen  specific  factors  that  contribute 
to  the  effectiveness  of  instruction  in  the  classroom  and 
which  can  be  improved  by  supervisory  assistance  a  second 
opinionnaire*  was  prepared  and  administered  prior  to  the 
1959  teacher  rating.  The  1957  Ideal  List  was  placed  on 
a  form  in  random  order  and  supervising  principals  were 
asked  to  rank  them  in  descending  order  of  importance. 

The  result  of  the  tabulation  of  submissions  of  this  form 
and  correlations  made  with  the  1957  list  were  indicative 
of  any  change  in  thinking  on  the  part  of  supervising 
principals . 

This  re-ranking  of  the  factors  also  served  as  a 
conditioner  for  principals  preparing  to  make  a  second 
rating  of  classroom  instruction. 

The  rank-difference  correlation  was  applied  to 
determine  the  change  and  to  substantiate  or  deny  the 
basic  assumption  that  no  significant  change  in  order  had 
taken  place. 
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VII.  A  SECOND  RUN 

The  principals’  second  rating  of  classroom  instruc¬ 
tion  utilized  the  same  rating  scale  employed  in  the  1957 
survey.  Newly  appointed  principals  we re  included  in  the 
second  run.  It  was  felt  that  the  gross  effect  of  such  a 
small  percentage  of  the  total  number  would  not  influence 
the  final  outcome.  Also  because  a  constant  turnover  of 
the  teaching  staff  existed  as  a  normal  problem  for  super¬ 
vising  principals  all  teachers  were  included  in  the 
second  rating. 


VIII.  THE  SAMPLE 

The  study  involved  all  supervising  principals  in 
the  Calgary  Public  Elementary  and  Junior  High  Schools 
and  all  teachers  under  their  supervision  who  were  teaching 
more  than  half  time. 

IX.  DISTRIBUTION  AND  COLLECTION  OF  THE  DATA 

The  distribution  of  the  Principals’  Opinionnaires 
and  the  Teacher  Rating  Scales  was  made  through  the 
Elementary  and  Junior  High  School  Principals’  Associations. 
Accompanying  each  distribution  a  careful  explanation  of 
directions  and  objectives  was  given  by  members  of  the 
evaluating  committee. 
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Collection  of  the  completed  forms  was  carried  out 
through  the  mails  by  the  Central-Office  clerical  staff. 
Complete  annonymity  was  maintained  in  every  case. 

The  tabulation  of  the  data  was  completed  by  the 
evaluation  committee  and  the  results  are  discussed  in  the 
following  chapter. 
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CHAPTER  IV 


TABULATION  AND  GENERAL  ANALYSIS 
OF  THE  RESULTS  OF  THE  EVALUATION 

Proceeding  with  the  experiment  on  evaluation, 
questionnaires  were  sent  out  and  completed  at  the  begin¬ 
ning  and  at  the  close  of  the  two-year  period.  An  analysis 
of  the  submissions  is  made  in  this  chapter. 

I.  THE  PRINCIPALS’  1957  OPINIONNAIRE 

Distribution  and  response .  The  Principals’  1957 
Opinionnaire*  was  distributed  to  the  fifty-nine  super¬ 
vising  principals  in  charge  of  Elementary  and  Elementary- 
Junior  High  Schools  asking  them  to  list  in  descending  order 
of  importance  sixteen  specific  factors  that  contribute  to 
the  effectiveness  of  instruction  in  the  classroom  and  which 
can  be  improved  by  supervisory  assistance.  Forty-nine  or 
83.1  Per  cent  of  the  principals  responded.  Such  a  response 
was  indicative  of  the  interest  and  cooperation  the  principals 
expressed  in  the  evaluation.  The  large  percentage  of 
returns  also  assured  the  committee  of  increased  validity 
for  the  Ideal  List  which  was  to  be  constructed  from  the 
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submissions.  Table  I  sets  forth  the  results  of  the  distri¬ 
bution  and  response  to  the  Opinionnaire. 

General  results.  In  the  Principals’  1957  Opinionnaire, 
principals  were  asked  to  list  in  descending  order  of 
importance  sixteen  specific  factors  that  contribute  to  the 
effectiveness  of  instruction  in  the  classroom  and  which 
can  be  improved  by  supervisory  assistance.  To  arrive  at 
a  consensus  of  opinion  the  lists  submitted  were  carefully 
examined  and  analyzed  by  the  panel  of  experts.  Little 
interpretation  by  the  panel  was  necessary  as  the  sub¬ 
missions  stated,  with  unmistakable  clarity,  the  contribu¬ 
ting  factors.  The  final  analysis  of  the  factors  indicated 
that  twenty-seven  were  submitted  for  consideration. 

A  frequency  distribution  of  choices  was  made.  In 
addition,  a  system  of  weighting  was  adopted  to  insure  that 
the  most  important  factors  chosen  were  given  recognition 
in  relation  to  the  less  important  ones.  This  was  done  by 
assigning  a  weight  of  eight  to  first  choices,  seven  to 
second,  six  to  third,  ...  ,  and  zero  to  all  beyond  the 
eighth.  The  final  score  for  each  factor  was  thus  determined 
and  the  first  sixteen  accepted  formed  the  Ideal  List.  The 
frequency  and  weight  for  each  is  given  in  Table  II,  page 
31.  From  this  table  of  results  the  rank  order  of  the 
sixteen  items  was  established  and  is  also  shown  in  Table  II. 


. 


.  '  i'jr  : 

I  (  .')■  uenoaea*! 


■  "  1  ‘  [-  .i':. 1 1 ■.  V'i: 


tl. 

.  3  ! 0 1  '  J  r' 


•J‘  ‘  "  '  I  .  U I'.  J"-; 

.  :  •  r, 


' 

•  ■  '  no ini  ©a 

.  3  1  :  'J 

-  •  0  '. 

.  o  {  s  '  '  .  t  ’ 

"  .  ■  -r. 

.  ,'r  '.  '■ . 

. 

rjQ  ..  ...  j 

.  ■  ' 

<  a  9  o  i  -  i  '  '■ 

t  ...  j  t  ; 

• 

,11  9 !  i  j  »vi  t  do  t  '  ,r  '  1 

1 

8  lie  .  '  :  -  '  8  I 


TABLE  I 


PRINCIPALS1  1957  QUESTIONNAIRE 


DISTRIBUTION 

STAGE 

AND  RESPONSE 

1  (57) 

Possible  Returns 

59 

Actual  Returns 

49 

Per  Cent  Returned 

63.1 

Returns  Rejected 

1 

Usable  Returns 

46 

Per  Cent  Usable 

61.4 

\Y  i  ’ 


:  ,t  ..  to 

. 

-  •  v  j  o;  • 

■  ; . . 

.  *  j  i... .  f 

. 

•  >1  c  -  t  o’ 

31 


TABLE  II 

1957  RANK  ORDER  OF  THE  SIXTEEN  SPECIFIC  FACTORS  THAT 
CONTRIBUTE  TO  THE  EFFECTIVENESS  OF  INSTRUCTION 
AS  DETERMINED  BY  FREQUENCY  AND  WEIGHT 


Factors 

f . 

Weight 

Total 

Rank 

Planning 

46 

271 

317 

1 

Individual  Differences 

47 

124 

171 

2 

Discipline 

33 

130 

16S 

3 

Curri culum 

30 

124 

154 

4 

Methods  and  Techniques 

36 

115 

151 

5 

Teacher 

33 

102 

135 

6 

Rapport 

26 

103 

129 

7 

Routines 

33 

SI 

119 

S 

Aids 

35 

33 

73 

9 

Evaluation  and  Reporting 

2S 

37 

65 

10 

Classroom 

21 

33 

59 

11 

Professional  Growth 

17 

35 

52 

12 

Pupil  Participation 

14 

36 

50 

13 

Staff  Relations 

13 

27 

40 

14 

Directives 

15 

23 

33 

15 

Supervision  of  Written  Work 

13 

22 

35 

16 
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Some  months  later  a  discussion  of  the  ordering  of 
the  sixteen  points  brought  to  light  the  fact  that  since 
all  principals  had  not  submitted  the  same  sixteen  points 
a  change  in  rank  might  have  occurred.  If  after  compiling 
the  list,  they  had  been  given  an  opportunity  to  list  the 
sixteen  accepted  a  new  order  may  have  resulted.  It  was 
recognized  that  such  a  procedure  would  have  improved  the 
validity  of  the  list  but  was  then  out  of  the  question. 

Since  this  list  was  basic  to  instruction  it  was 
assumed  that  it  would  not  change  substantially  over  the 
two-year  period.  To  validate  this  hypothesis  it  was 
decided  that  a  re-ordering  of  the  Ideal  List  would  be 
requested  in  1959  and  the  results  dealt  with  at  that  time. 

As  planned,  the  format  of  the  Teacher  Rating  Scale 
was  constructed  from  the  established  Ideal  List. 

II.  THE  1957  RATING  OF  TEACHERS 

The  instrument  to  be  used  for  the  rating  of  class¬ 
room  instruction  as  it  existed  at  the  outset  of  the 
experiment  was  distributed  to  supervising  principals  in 
December  of  1957  to  be  returned  by  the  end  of  February 
1956.* 

Distribution  and  response .  Of  a  possible  return  of 
716  ratings  675  or  94.3  per  cent  were  received.  Of  these, 


*Appendix — pp.  55-56. 
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thirty  were  rejected  because  of  a  repetition  of  factor 
numbers  in  the  ratings.  Table  III  summarizes  the  results 
of  the  distribution  and  response. 

General  results .  A  tabulation  of  the  645  usable 
ratings  divulged  the  rank  order  of  the  sixteen  factors  in 
1957.  To  determine  the  relative  positions  of  the  character¬ 
istics  each  first  choice  was  given  a  count  of  one  and  a 
weight  of  seven,  each  second  and  third  choices  a  count  of 
one  and  weight  of  six,  and  so  on.  The  final  total  score 
for  each  factor  determined  its  position  in  the  rating. 

The  results  are  shown  in  Table  IV,  page  35. 

A  rank-difference  correlation  based  on  the  Ideal 
List  ordering  and  the  rank  order  indicated  by  the  Teacher 
Rating  was  calculated.  The  result,  .41,  (not  significant 
at  .05  level),  indicated  that  in  the  opinion  of  supervising 
principals  the  instruction  being  offered  in  the  classrooms 
as  a  whole  did  not  substantially  parallel  what  the  princi¬ 
pals  judged  good  classroom  instruction  to  be. 

III.  RANK  ORDER  REVISION  OF  THE  1957  IDEAL  LIST 

After  the  two-year  period  data  was  again  gathered. 

Activities  in  the  intervening  period.  In  this  period 
a  variety  of  planned  and  unplanned  activities  took  place. 
In-service  training  courses,  discussions  in  the  Supervising 
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TABLE  III 


PRINCIPALS’  1957  TEACHER  RATING  SCALE 
DISTRIBUTION  AND  RESPONSE 
STAGE  2  (57) 


Possible  Returns  716 

Actual  Returns  675 

Per  Cent  Returned  94*3 

Returns  Rejected  30 

Usable  Returns  645 

90.1 


Per  Cent  Usable 
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TABLE  IV 

1957  RANK  ORDER  OF  THE  SIXTEEN  SPECIFIC  FACTORS  THAT 
CONTRIBUTE  TO  THE  EFFECTIVENESS  OF  INSTRUCTION 
AS  DETERMINED  BY  TEACHER  RATINGS 


Factors 


Teacher  Rating  (57) 


Classroom  14 
Directives  $ 
Teacher  1 
Rapport  2 
Discipline  3 
Routines  10 
Planning  4 
Individual  Differences  12 
Pupil  Participation  6 
Supervision  of  Written  Work  7 
Evaluation  and  Reporting  13 
Professional  Growth  15 
Staff  Relations  11 
Curriculum  9 
Methods  and  Techniques  5 
Aids  16 
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Principals’  Association  meetings,  attendance  at  University, 
participation  in  discussion  groups,  professional  reading, 
and  staff  meetings  are  examples.  Because  of  the  foregoing 
the  committee  felt  that  principals  might  have  emerged  with 
a  new  concept  of  the  rank  order  of  the  Ideal  List.  Also,  as 
stated  on  page  32,  the  validation  of  the  hypothesis  that  no 
substantial  change  in  order  was  to  be  substantiated.  There¬ 
fore,  the  original  list  of  sixteen  characteristics  of 
effective  instruction  was  distributed  for  re-ordering  by 
the  principals  in  November  1959.  The  form  used  is  found 
on  pages  57-58  in  the  Appendix. 

Distribution  and  response .  Of  the  eighty  tests 
distributed  sixty-eight  or  35  per  cent  were  returned. 
Sixty-seven  or  33.3  per  cent  of  the  possible  returns  were 
completed  correctly.  Comparative  figures  for  1957  and  1959 
are  summarized  in  Table  V. 

General  re suits.  Applying  the  same  system  of 
frequency  count  and  weights  as  used  in  the  original 
tabulation  the  results  as  shown  in  Table  VI,  page  38,  were 
obtained.  For  the  purpose  of  comparison  both  the  original 
1957  ideal  rank  order  and  the  1959  order  are  given. 

A  rank-difference  correlation  between  the  1957 
Ideal  List  ranking  and  the  re-ranking  submitted  in  1959 
resulted  in  an  r  equal  to  .65  (significant  at  the  .01 
level).  This  calculation  was  interpreted  as  evidence 
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COMPARISON  OF 
QUESTIONNAIRES , 

TABLE  V 

PRINCIPALS1  1957 
DISTRIBUTION  AND 

AND  1Q59 
RESPONSE 

Stage  1  (57) 

Stage  1  (59) 

Possible  Returns 

59 

SO 

Actual  Returns 

49 

6S 

Per  Cent  Returned 

S3.1 

S5.0 

Returns  Rejected 

1 

1 

Usable  Returns 

4S 

67 

Per  Cent  Usable 

SI. 4 

83.8 
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COMPARISON  OF  1957  AND  1959  RANK  ORDERS  OF  THE  SIXTEEN 
SPECIFIC  FACTORS  THAT  CONTRIBUTE  TO  THE 
EFFECTIVENESS  OF  INSTRUCTION 


Factors 

Ideal  Rank 
(57) 

Ideal  Rank 
(59) 

Classroom 

11 

13 

Directives 

15 

10 

Teacher 

6 

2 

Rapport 

7 

4 

Discipline 

3 

1 

Routines 

8 

11 

Planning 

1 

3 

Individual  Differences 

2 

7 

Pupil  Participation 

13 

8 

Supervision  of  Written  Work 

16 

9 

Evaluation  and  Reporting 

10 

14 

Professional  Growth 

12 

12 

Staff  Relations 

14 

15 

Curriculum 

4 

5 

Methods  and  Techniques 

5 

6 

Aids 

9 

16 
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that  the  1957  ordering  of  the  sixteen  factors  had  not 
changed  greatly  in  1959. 

IV.  THE  1959  RATING  OF  TEACHERS 

In  September  1959  all  supervising  principals  in  the 
Elementary  and  Junior  High  Divisions  of  the  school  system 
were  advised  that  the  second  rating  of  teachers  for  the 
evaluation  of  supervision  was  to  be  made.  This  advance 
notice  was  given  so  that  principals  would  be  thinking 
about  the  rating  and  would  organize  their  supervisory 
programs  accordingly. 

The  teacher  rating  forms  were  distributed  in 
December  to  be  returned  by  the  end  of  February.  Since 
this  was  not  to  be  used  as  a  measure  of  individual  teachers 
all  ratings  again  remained  annonymous.  And,  as  was 
stated  previously,  the  rank  order  of  neither  the  1957  nor 
the  revised  1959  Ideal  Lists  were  revealed  to  the  principals. 

Distribution  and  response .  Of  944  teacher  rating 
forms  distributed  932  or  9$.  7  per  cent  were  returned.  Of 
those  returned  thirty-seven  had  to  be  rejected  because  of 
number  duplications  in  the  ratings.  Therefore,  $95  or 
94. $  per  cent  of  the  possible  were  completed  correctly. 

Table  VII  summarizes  comparative  figures  for  the  1957  and 
1959  distribution  and  response  to  the  teacher  ratings. 
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General  results.  A  tabulation  of  the  completed 
1959  forms  by  frequency  count  and  weighting  revealed  the 
factor  order  shown  in  Table  VIII,  page  42.  The  rank  order 
of  factors  determined  from  the  1957  teacher  rating  is  also 
included. 

The  calculation  of  the  rank-difference  correlation 
between  the  ordering  obtained  from  the  1959  teacher  rating 
and  the  1959  revised  order  of  the  Ideal  List  produced  an 
r  equal  to  .87  (significant  at  the  .01  level).  This 
correlation  was  interpreted  as  evidence  that  the  relation 
between  what  was  expected  in  classroom  instruction  and 
what  existed  at  this  time  showed  a  very  high  relationship. 

V.  RESULTS  OF  ANALYSIS 

The  rank-difference  correlation  between  the  1957 
Ideal  List  ranking  and  the  1957  Teacher  Rating  ranking 
of  .41  indicated  that  principals  did  not  find  in  the 
classroom  instruction  what  they  thought  they  should.  A 
similar  comparison  made  in  1959  between  the  1959  Ideal 
List  and  the  1959  Teacher  Rating  resulted  in  a  correlation 
of  .87  which  indicated  that  considerable  unanimity  of 
thought  existed.  The  size  of  the  change  in  the  cor¬ 
relations,  it  was  felt,  supported  the  original  hypothesis. 

Using  the  1957  Ideal  List  as  the  standard  further 
support  for  the  hypothesis  was  gained.  The  1957  Ideal  List 
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TABLE  VII 

COMPARISON  OF  PRINCIPALS T  1957  AND  1959 
RATING  SCALES,  DISTRIBUTION  AND  RESPONSE 

Stage  2  (57)  Stage  2  (59) 

Possible  Returns  716  944 

Actual  Returns  675  932 

Per  Cent  Returned  94.3  93.7 

Returns  Rejected  30  37 

Usable  Returns  645  395 

90.1  94.3 


Per  Cent  Usable 
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TABLE  VIII 

COMPARISON  OF  1957  AND  1959  RANK  ORDERS  OF  THE 
SIXTEEN  SPECIFIC  FACTORS  THAT  CONTRIBUTE  TO 
THE  EFFECTIVENESS  OF  INSTRUCTION 


Factors 

Teacher 

Rating 

(57) 

Teacher 

Rating 

(59) 

Ideal 

Rank 

(57) 

Ideal 

Rank 

(59) 

Classroom 

14 

14 

11 

13 

Directives 

a 

11 

15 

10 

Teacher 

i 

1 

6 

2 

Rapport 

2 

3 

7 

4 

Discipline 

3 

2 

3 

1 

Routines 

10 

10 

a 

11 

Planning 

4 

4 

1 

3 

Individual  Differences 

12 

a 

2 

7 

Pupil  Participation 

6 

6 

13 

8 

Supervision  of  Written  Work 

7 

7 

16 

9 

Evaluation  and  Reporting 

13 

13 

10 

14 

Professional  Growth 

15 

15 

12 

12 

Staff  Relations 

11 

12 

14 

15 

Curriculum 

9 

9 

4 

5 

Methods  and  Techniques 

5 

5 

5 

6 

Aids 

16 

16 

9 

16 
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showed  a  correlation  of  .41  with  the  1957  Teacher  Rating 
and  a  .5$  correlation  with  the  1959  Teacher  Rating. 

Support  for  the  hypothesis  was  lacking  when  the 
revised  1959  Ideal  List  was  employed  as  the  standard. 

The  correlation  between  the  1957  Teacher  Rating  and  the 
1959  Ideal  List  was  .$7  which  was  identical  with  the 
correlation  between  the  1959  Teacher  Rating  and  the  1959 
Ideal  List. 

It  was  concluded  that  although  changes  either  in 
the  thinking  on  the  part  of  principals  or  the  effective¬ 
ness  of  the  instruction  had  taken  place  the  measures 
obtained  failed  to  pin-point  this  with  any  degree  of 


accuracy. 
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CHAPTER  V 


SUMMARY,  CONCLUSIONS,  AND  RECOMMENDATIONS 

In  this  chapter  the  conclusions  and  recommendations 
follow  a  restatement  of  the  problem,  sources  of  data, 
and  the  procedure. 

I.  RESTATEMENT  OF  THE  PROBLEM 

The  purpose  of  this  study  was  to  evaluate  the 
de-centralized  plan  of  supervision  practiced  in  the 
Calgary  Public  Elementary  and  Junior  High  Schools 
during  the  period  1957  to  1959.  The  main  variable 
considered  was  time  free  for  supervision  as  granted 
by  the  Board  in  1957.  Throughout  the  evaluation  the 
question  to  be  answered  was:  Is  time  free  from  teaching 
duties  for  supervising  principals  effective  in  im¬ 
proving  classroom  instruction? 

II.  SOURCES  OF  DATA 

The  subjects  from  which  the  data  for  this  study 
were  drawn  included  all  supervising  principals  in  the 
Elementary  and  Junior  High  Schools  and  their  teaching 
staffs . 
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III.  PROCEDURE 


At  the  outset  an  Ideal  List  of  sixteen  specific 
factors  that  contribute  to  the  effectiveness  of  instruc¬ 
tion  was  established.  The  factors  submitted  were  limited 
to  those  which  can  be  improved  by  supervisory  assistance. 
The  submissions  were  made  by  the  principals  who  had  been 
granted  released  time  for  supervision. 

These  factors  were  then  adapted  to  a  Teacher 
Rating  Scale  to  ascertain  the  existing  state  of  classroom 
instruction. 

After  a  two-year  period  a  re-ranking  of  the  Ideal 
List  was  established  and  a  second  rating  of  teachers  made. 

Calculations  of  rank-difference  correlations 
between  the  rank  orders  established  in  the  Ideal  Lists 
and  those  determined  from  the  Teacher  Ratings  were  made  to 
find  the  growth  in  effectiveness  of  supervision. 

IV.  CONCLUSIONS 

On  the  basis  of  the  results  which  were  obtained 
definite  growth  was  evident.  The  correlation  found  to 
exist  in  1957  as  compared  with  that  found  in  1959  indi¬ 
cated  that  a  considerable  growth  toward  unanimity  of 
thought  between  principals  and  teachers  had  taken  place. 

Recognizing  that  the  controlling  variable  was 
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assumed  to  be  time  free  for  supervision  it  must  be  em¬ 
phasized  that  many  other  variables  were  influencing  both 
principals  and  teachers.  The  formal  and  informal  dis¬ 
cussions  of  the  Ideal  Lists  appeared  to  have  the  effect  of 
making  those  concerned  more  conscious  of  what  was  expected 
of  them.  Indications  were  also  observed  that  these 
discussions  tended  to  consolidate  the  thinking  on  where 
emphasis  should  be  placed.  Nor  can  it  be  overlooked  that 
the  in-service  training  of  principals  by  the  central-office 
supervisory  staff  would  lead  to  either  direct  or  indirect 
suggestions  as  to  where  emphasis  should  or  would  be 
expected  to  be  placed.  The  fact  that  principals  knew  that 
an  evaluation  of  their  programs  was  being  made  prompted 
considerable  thought,  discussion,  and  supervisory  action. 
Nevertheless,  without  time  for  the  above  activities  the 
degree  of  growth  would,  it  is  believed,  be  considerably 
handicapped. 


V.  APPLICATION 

Studies  of  this  nature,  it  is  agreed,  have  many 
desirable  outcomes.  Quantitative  support  is  given  to 
particular  practices.  Much  interest  is  created  among 
principals  and  teachers  in  the  results  of  their  efforts. 
The  dynamic  effect  upon  the  morale  and  professional  growth 
of  staffs  is  very  rewarding. 
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The  methods  and  procedures  used  in  this  study  are 
adaptable  to  any  system  utilizing  or  contemplating  a 
scheme  of  de-centralized  supervision. 

VI.  RECOMMENDATIONS 

Arising  from  the  conclusions  certain  recommen¬ 
dations  appear  justified. 

1.  The  de-centralized  scheme  of  supervision  should 
be  continued  and  expanded. 

I 

2.  An  investigation  into  in-service  methods  of 
preparing  principals  for  their  jobs  could  be  used  to 
build  a  training  program  for  them. 

3.  Continuation  of  the  discussion  and  study  of 
supervisory  practices  would  tend  to  improve  and  stabilize 
supervision. 

4.  A  repetition  of  the  above  evaluation  would  have 
the  desirable  effect  of  keeping  the  system  alert  to  the 
accepted  characteristics  of  good  classroom  instruction 
and  supervision. 

5.  Further  studies  which  inform  teachers  and 
principals  of  the  results  of  their  efforts  would  be 
professionally  desirable. 
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VII .  NEEDED  RESEARCH 


In  conclusion  a  number  of  suggestions  are  made  for 
further  research. 

1.  The  conducting  of  a  similar  study  in  another 
city  and  comparing  results  could  lead  to  a  refining  of  the 
instruments  and  aid  substantially  in  establishing  a  needed 
measuring  stick  for  supervision. 

2.  Further  studies  could  endeavor  to  measure  the 
effect  of  any  concentrated  effort  on  the  part  of  adminis¬ 
tration  to  improve  supervisory  practices. 

3.  The  repetition  of  this  study  at  a  later  date 
would  reveal  changes  in  emphasis  held  by  supervisors  and 
thus  determine  future  action  in  programs  of  supervision 
and  in-service  training. 

A.  Future  studies  should  investigate  and  compare 
the  opinions  of  teachers  and  supervisors  on  what  consti¬ 
tutes  an  ideal  list  of  characteristics  of  effective  class¬ 
room  instruction. 

5.  A  study  of  pupil  achievement  in  situations  where 
time  is  given  for  supervision  and  where  it  is  not  would 
further  measure  the  value  of  supervision. 
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6.  A  comparison  of  the  cost  of  de-centralized 
supervision  as  compared  with  that  of  centralized  super¬ 
vision  would  be  a  study  of  considerable  interest  to  school 
boards. 
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STAGE  ONE  OF  EVALUATION  OF  SUPERVISION 


October,  1957. 

Please  list  in  descending  order  of  importance  16 
specific  factors  that  contribute  to  the  effectiveness  of 
instruction  in  the  classroom  and  which  can  be  improved  by 
supervisory  assistance. 

To  be  submitted  to  Mr.  L.  A.  Daniels  on  or  before 
October  31st,  1957. 
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MODIFIED  RATING  SCALE  OF  EMPLOYEE  CHARACTERISTICS 


(Safran  Index) 


This  is  in  a  sense  a  rating  scale.  You  are  being 
asked  to  give  your  honest  opinion  of  how  well  certain 
statements  apply  to  the  individual  being  rated.  Read  all 
the  statements,  then  place  the  number  of  each  statement 
in  one  of  the  little  squares  which  appear  below,  to  show 
how  well  the  statement  applies.  In  the  top  square,  place 
the  number  of  the  statement  that  applies  very  well  to  the 
person  being  rated.  In  the  next  two  squares  place  the 
numbers  of  the  two  statements  that  apply  well  to  the 
person,  and  so  on.  You  may  fill  in  the  squares  in  any 
order  you  wish.  Check  off  each  statement  as  you  put  its 
number  in  the  square.  Be  sure  that  only  one  number  appears 
in  each  square  and  that  this  number  is  not  duplicated  in 

any  other  square. 


Statements 


(1)  Works  with  great  enthusiasm 

(2)  Usually  lets  others  take  the  lead 

(3)  Works  well  with  others 

(4)  Unquestionably  dependable 

(5)  Needs  to  be  constantly  supervised 

(6)  An  excellent  leader 

(7)  Rarely  shows  enthusiasm 

(8)  Uses  his  head  on  the  job 

(9)  Seeks  and  sets  for  himself  new  tasks 

(10)  Dependability  doubtful 

(11)  Rarely  completes  work 

(12)  Finds  it  hard  to  work  with  others 

(13)  Rarely  uses  his  head  on  the  job 

(14)  Works  well  without  supervision 

(15)  Needs  considerable  prodding 

(16)  Usually  finishes  anything  he  undertakes 


Statement 

Statement 

Statement 

Statement 

Statement 

Statement 

Statement 


applies  very  well 
applies  well 

applies  somewhat  (or  quite)  well 
may  or  may  not  apply 
applies  somewhat  (or  quite)  poorly 
applies  poorly 
applies  very  poorly 
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CALGARY  SCHOOL  BOARD 

STAGE  2  OF  EVALUATION  OF  SUPERVISION  1957-53 


Stateme nts:  (These  are  not  listed  in  order  of  importance.) 


1.  CLASSROOM 

2.  DIRECTIVES 

3.  TEACHER 

4.  RAPPORT 

5.  DISCIPLINE 

6.  ROUTINES 

7.  PLANNING 

8.  INDIVIDUAL 
DIFFERENCES 

9.  PUPIL 
PARTICIPATION 

10.  SUPERVISION  OF 
WRITTEN  YORK 

11.  EVALUATION 
And  REPORTING 


The  teacher  attends  adequately  to  the 
physical  comfort  of  children  in  the 
classroom. 

The  teacher  follows  procedures  set 
forth  in  bulletins,  handbooks,  and 
outlines  which  are  provided. 

The  teacher  possesses  many  personal 
characteristics  which  contribute  to 
the  success  of  the  classroom. 

The  teacher  develops  a  warm  friendly 
relationship  with  the  pupils. 

The  teacher  maintains  the  type  of  class 
control  that  results  in  a  good 
learning  situation. 

The  teacher  establishes  effective 
time-saving  classroom  routines. 

The  teacher  plans  work  carefully  in 
relation  to  both  long-range  and 
short-range  objectives. 

The  teacher  understands  child  growth 
and  teaches  to  meet  individual  and 
group  needs. 

The  teacher  ensures  pupil  participation 
appropriate  to  the  objectives  as  an 
integral  part  of  instruction. 

The  teacher  attends  carefully  and 
systematically  to  the  assignment, 
supervision,  and  checking  of  written 
work. 

The  teacher  evaluates  and  reports 
pupil  achievement  in  ways  which  are 
helpful  to  pupils  and  parents. 
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12. 

PROFESSIONAL 

GROV/TH 

The  teacher  engages  in  activities 
conducive  to  professional  growth. 

13. 

STAFF  RELATIONS 

The  teacher  works  effectively  and 
cooperatively  with  other  staff 
members. 

14. 

CURRICULUM 

The  teacher  knows  thoroughly  the 
objectives  and  content  of  the  curri¬ 
culum  appropriate  to  the  grade  and 
subject  taught. 

15. 

METHODS  AND 

TECHNIQUES 

The  teacher  uses  methods  and  tech¬ 
niques  of  instruction  appropriate 
to  the  group  or  subject  being  taught. 

• 

vO 

rH 

AIDS 

The  teacher  uses  efficiently  teaching 
aids  available  in  the  school. 

Statement 

is 

Statement 

is 

Statement 

is 

Statement 

is 

Statement 

is 

Statement 

is 

Statement 

is 

most  applicable 
very  applicable 
applicable 

moderately  applicable 
less  applicable 
still  less  applicable 
least  applicable 
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CALGARY  SCHOOL  BOARD 

November,  1959. 

STAGS  ONE  (59)  OF  EVALUATION  OF  SUPERVISION 
Did  you  participate  as  a  principal  in  this  experiment  in 

1957?  _ 

Answer  Yes  or  No. 

Please  number  in  descending  order  of  importance  these 
16  specific  factors  that  contribute  to  the  effectiveness  of 
instruction  in  the  classroom  and  which  can  be  improved  by 
supervisory  assistance.  These  are  not  necessarily  listed  in 
any  order  of  importance.  Place  a  1  in  the  square  opposite 
the  factor  which  you  judge  to  be  the  most  important;  place  a 
2  in  the  square  opposite  the  factor  which  you  consider  second 
in  importance,  and  so  on  down  to  a  16  opposite  what  you 
judge  to  be  the  least  important  factor. 

To  be  submitted  to  Mr.  L.  A.  Daniels  on  or  before 
November  24th,  1959. 


CLASSROOM 

The  teacher  attends  adequately  to  the 
physical  comfort  of  children  in  the 
classroom. 

DIRECTIVES 

The  teacher  follows  procedures  set 
forth  in  bulletins,  handbooks,  and 
outlines  which  are  provided 

TEACHER 

The  teacher  possesses  many  personal 
characteristics  which  contribute  to 
the  success  of  the  classroom. 

RAPPORT 

The  teacher  develops  a  warm  friendly 
relationship  with  the  pupils 

DISCIPLINE 

The  teacher  maintains  the  type  of 
class  control  that  results  in  a  good 
learning  situation. 

ROUTINES 

The  teacher  establishes  effective 
time-saving  classroom  routines. 

PLANNING 

The  teacher  plans  work  carefully  in 
relation  to  both  long-range  and 
short-range  objectives. 
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INDIVIDUAL 

DIFFERENCES 


PUPIL 

PARTICIPATION 


SUPERVISION  OF 

WRITTEN  WORK 


EVALUATION 
AND  REPORTING 


PROFESSIONAL 

GROWTH 


STAFF 

RELATIONS 


CURRICULUM 


METHODS  AND 

TECHNIQUES 


AIDS 


The  teacher  understands  child  growth 
and  teaches  to  meet  individual  and 
group  needs. 

The  teacher  ensures  pupil  participa¬ 
tion  appropriate  to  the  objectives  as 
an  integral  part  of  instruction. 

The  teacher  attends  carefully  and 
systematically  to  the  assignment, 
supervision,  and  checking  of  written 
work. 

The  teacher  evaluates  and  reports 
pupil  achievement  in  ways  which  are 
helpful  to  pupils  and  parents. 

The  teacher  engages  in  activities 
conducive  to  professional  growth. 

The  teacher  works  effectively  and 
cooperatively  with  other  staff 
members. 

The  teacher  knows  thoroughly  the 
objectives  and  content  of  the  curri¬ 
culum  appropriate  to  the  grade  and 
subject  being  taught. 

The  teacher  uses  methods  and  tech¬ 
niques  of  instruction  appropriate  to 
the  group  or  subject  being  taught. 

The  teacher  uses  efficiently  teaching 
aids  available  in  the  school. 
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